Improving the Robustness of Relevance-Based Language Models

We propose a new robust relevance model that can be applied to both pseudo feedback and true relevance feedback in the language-modeling framework for document retrieval. There are three main differences between our new relevance model and the Lavrenko-Croft relevance model. First, a query is treated as a short, special document and included in approximating a relevance model, in addition to a number of top ranked documents returned from the first round retrieval for pseudo feedback, or a number of relevant documents for true relevance feedback. Second, instead of using a uniform prior as in the original relevance model, documents are assigned with different priors according to their lengths (in terms) and ranks in the first round retrieval. Third, the probability of a term in the relevance model is further adjusted by its probability in the background language model. We have applied the proposed new model to both pseudo feedback and true relevance feedback. In both cases, we have compared the performance of our model to that of the two baselines: the original relevance model and a linear combination model. Experiments were carried out with TREC title queries 101 to 200 on AP collections and queries 301 to 400 on a heterogeneous collection consisting of the data from TREC disk 4 and 5. The results show that the proposed new model outperforms both of the two baselines – the original relevance model and a linear combination model -in terms of mean average precision. Furthermore, for pseudo feedback, it is less sensitive to the number of documents than the two baseline models, and for true relevance feedback, it achieves better performance than the two baseline models using fewer relevant documents.